Enhancing Outfit Recommendation with a CNN-kNN Hybrid Model and Digital Wardrobe Management

Abstract

In the age of looking good, we come across that one question everyday; What should I wear to look good today?. The question span a broad variety of topics, such as what am I missing in wardrobe? what is the current trend? will this suit my skin tone? will this work in current climate. the integration of technology and clothing has revolutionized the way individuals perceive and choose their outfits. This study presents a cutting-edge outfit recommendation system that combines the analysis of user skin tones, real-time weather data, and advanced machine learning algorithms. The system utilizes a digital wardrobe to store a vast array of fashion items and employs a Convolutional Neural Network (CNN) to extract intricate features from these items, enhancing the accuracy of clothing recognition. To personalize recommendations, the system incorporates user skin tone analysis, ensuring that the suggested outfits complement the wearer\'s complexion. Additionally, real-time weather data is integrated to align outfit suggestions with the prevailing weather conditions, ensuring both style and comfort. The system utilizes K-Means clustering to categorize users based on similar fashion preferences and Decision Tree algorithms to refine outfit recommendations further. The proposed system not only enhances user experience but also contributes to sustainable fashion practices by promoting the optimal utilization of existing wardrobes, reducing impulsive shopping, and minimizing fashion waste. Through rigorous testing and validation, the system demonstrates its effectiveness in providing tailored outfit recommendations, thereby reshaping the future of personalized fashion choices

Introduction

I. INTRODUCTION

The world of fashion is a realm of self-expression and personal style, where clothing choices often serve as a reflection of one's identity and preferences. As the fashion industry rapidly integrates with the digital landscape, outfit recommendation systems have emerged as a powerful ally for fashion enthusiasts and shoppers. These systems leverage cutting-edge technologies to provide users with tailored suggestions, making it easier to discover the perfect ensemble for any occasion. In recent years, research and development in outfit recommendation systems have witnessed significant advancements, driven by the fusion of data-driven approaches and deep learning techniques. This introduction provides an overview of the burgeoning field of outfit recommendation systems, emphasizing the pivotal role that references [11] to [16] have played in shaping its trajectory. Reference [11], authored by Y Qian, P Giaccone, M Sasdelli, and E Vazquez in 2017, explores the concept of "Algorithmic Clothing: Hybrid Recommendation, from Street-Style-to-Shop." This work delves into the intricacies of bridging the gap between street-style fashion and online shopping, offering a holistic view of how data-driven algorithms can assist users in translating their fashion inspirations into real-world outfits. Additionally, Elleuch, Mezghani, Khemakhem, and Kherallah's study, referenced as [12], published in 2021, underscores the significance of deep learning and transfer learning in "Clothing Classification using Deep CNN Architecture." This reference accentuates the role of convolutional neural networks (CNNs) in understanding clothing items and styles, paving the way for more accurate outfit recommendations based on visual cues. Furthermore, the work of Gupta, Agarwal, and Dave ([13]) in 2015 introduces an "Apparel Classifier and Recommender using Deep Learning," demonstrating the potential of deep learning to decipher and recommend clothing items based on their attributes. This reference highlights the foundational use of deep learning in the apparel domain. Bossard, Dantone, Leistner, Wengert, Quack, and Van Gool's contribution in 2012 ([14]), titled "Apparel Classification with Style," brings attention to the fusion of apparel classification and style analysis. The research delves into the intricacies of identifying clothing items and their stylistic attributes, a vital aspect of outfit recommendations. The breakthrough by Krizhevsky, Sutskever, and Hinton in 2012 ([15]) with "ImageNet classification with deep convolutional neural networks" laid the foundation for CNN-based deep learning, which has revolutionized image classification and recognition.

This advancement has significantly impacted the accuracy of outfit recommendation systems by enhancing the interpretation of fashion-related images. In the context of these seminal references and the broader landscape of outfit recommendation systems, this article delves into the evolving methodologies, challenges, and future prospects in the domain of fashion recommendation. By combining the insights from these references with recent developments, this work aims to contribute to the ongoing evolution of outfit recommendation systems, fostering a more personalized and delightful fashion experience for users.

II. MOTIVATION

In today's fast-paced and digitally connected world, fashion and personal style have become integral components of self-expression and identity. What we wear not only reflects our individual tastes but also plays a pivotal role in shaping the way we are perceived by others. Given the diverse occasions and events that populate our lives, from casual outings to formal gatherings, individuals often find themselves in need of guidance when it comes to selecting the perfect outfit. It is in this context that the development of an outfit recommendation system based on occasions emerges as a compelling and relevant pursuit.

III. LITERATURE SURVEY

Fashion recommender system using deep learning Angel [5] arul Jyothi,Razia Sulthana The deep learning techniques used by researchers to build the fashion recommenders were neural networks, Siamese networks, CNN, Autoen coders, GAN, R-CNN, and LSTM. While the recommender systems that were built were novel and robust, they have some limitations

Journal of emerging echnologies and innovative research Prof. Meera Sawalka, Aarti bobade Recommendation systems have the potential to explore new opportunities for retailers by enabling them to provide customized recommend dations to consumers based on information retrieved from the Internet. Study of Al driven Fashion Recommender system Hamam Mokayet, Hum yan chai Answering what distinguishes the fashion domain from that of other recommender systems leads to identifying fashion domain peculiarities. The main reasons why generic recommender systems cannot meet the needs in the FRS domain are: first, the subtle and subjective nature of fashion to be understood Fashion pairing recommendation system Miura,Yamasaki, Aizawa Assists users in determining what kind of coordinates are appropriate for their clothing and in purchasing appropriate items. Smart or intelligent recommendation Congying Guan, Sheng-Feng Qin Produces highly accurate prediction on apparel

IV. SYSTEM ARCHITECTURE

1. Data Acquisition

Users build a digital wardrobe by providing images or descriptions of their clothing items.

The system extracts garment attributes from the data (e.g., color, style, brand).

2. Feature Extraction

Machine learning techniques, like Convolutional Neural Networks (CNNs) process garment images to automatically extract features like color, pattern, and texture.

3. Event Recognition

The system takes user input or integrates with calendar apps to recognize the context of the outfit (e.g., formal event, casual outing).

4. Recommendation Algorithm

A recommendation algorithm considers various factors to generate outfit suggestions:

Garment attributes (extracted features)

User preferences learned from interactions with the system

Compatibility rules (e.g., style harmony between garments)

Event context (formal vs casual)

5. User Interface

The system displays outfit recommendations to the user through a user-friendly interface.

Users can interact with the recommendations, providing feedback or refining their preferences.

A. User-End Steps

1. User Registration and Login

Users register or log in using credentials(eg.gender, weather)

2. Profile Creation

Users create profiles, adding details like age, gender, and style preferences

Optionally, users upload pictures for a personalized experience

3. Adding Clothing Items

Users add items to their wardrobe by uploading pictures or providing details (category, colour, brand, etc.).

Implement an intuitive interface for easy item addition

4. Item Categorization and Tagging

Users categorize items (e.g., shirts, pants) and add tags (e.g., formal, casual) for easy searching.

Implement drag-and-drop or selection features for efficient categorization

5. Virtual Try-On (Optional)

Users can try on virtual outfits using augmented reality technology

Implement AR features for realistic outfit simulations[1].6. Outfit Creation and Recommendations:

Users mix and match items to create outfits

Provide suggestions based on fashion trends, occasion, and user preferences.

V. DATSET DESCIPRTION

1. Size: With 1346 rows, the dataset has a decent size for initial training. However, the effectiveness of machine learning models often improves with more data.
2. Columns: The five columns provide valuable information for understanding and classifying clothing items:
3. Date: This helps track when images were added, potentially useful for analysing trends or identifying outdated styles.
4. Type: With three main categories (casual, formal, and beachwear), this provides a good foundation for broader classification. The additional weather subcategories (e.g., summer, winter) further refine the classification based on seasonal suitability.
5. Gender: Separating clothing by gender allows for more targeted recommendations and analysis of fashion trends within each category.
6. Weather: Understanding the weather suitability of garments is crucial for outfit recommendation systems and can be used to personalize suggestions based on location and climate.
7. Path: The file paths ensure the model can access the corresponding image data for learning features and classifications.

A. Potential Uses

This dataset could be used for various purposes, including:

1. Training a Clothing Classification Model: The model could learn to automatically categorize new clothing images based on the provided categories and subcategories.
2. Developing an Outfit Recommendation System: By considering user preferences, weather conditions, and the information in the dataset, the system could recommend outfits that are both stylish and suitable for the occasion.
3. Analysing Fashion Trends: Examining the distribution of clothing types and weather categories within the dataset might reveal trends in user preferences or seasonal variations in clothing choices.

Overall, this well-structured dataset provides a solid foundation for exploring various applications in the realm of clothing classification and potentially outfit recommendation.

B. Backend Workflow

1. User Authentication and Authorization

Implement user authentication using JWT tokens for secure login

Authorize users based on their roles for different functionalities

2. Data Storage

Use a database system (e.g., PostgreSQL, MongoDB) to store user profiles, clothing items, and outfit data.

Utilize cloud storage (e.g., Amazon S3) for storing images

3. Image Processing for Skin Tone Analysis

Utilize image processing libraries like OpenCV and scikit-image to analyse the user's skin tone when an image is uploaded.

4. Outfit Recommendation Algorithm

Use machine learning libraries like scikit-learn and TensorFlow for building recommendation algorithms [5].

Implement algorithms considering occasion, user preferences, and skin tone analysis results to suggest outfits [2].

5. Integration with Retail APIs (Optional)

Integrate with external APIs from clothing retailers to fetch product information, prices, and availability.

# Technologies and Python Libraries:

1. Backend Framework: Flask or Django[1].

2. Database: PostgreSQL or MongoDB[3].

3. Image Processing: OpenCV, scikit-image[4].

4. Machine Learning: scikit-learn, TensorFlow[5].

5. Authentication: JSON Web Tokens (JWT)[1].

6. Cloud Storage:Amazon S3[3].

VI. RESULTS

User enters the event type, weather condition as an input to model1.Base on the input conditions Model1 creates “test_img_path” as the output.The output from Model1 serves as input for Model2. Here, the kNN algorithm searches the user's digital wardrobe for similar images, creating a more personalized selection.

Conclusion

We did research about similar models which lead to conclusion that there doesn’t exist any system which uses digital wardrobe to recommend outfits based on particular event and weather We built an appropriate dataset based on models requirement We build two models which work on algorithms like CNN and KNN We tested the model for different inputs, in conclusion model recommends similar outfits based on given input

References

[1] Wikipedia “Recommendation System”. [2] Q.T.J.L. Jia Jia, Ke Gao, “Trip outfits advisor: location -oriented clothing recommendation,” IEEE publication, vol. 19, pp. 1–19, 2016. [3] T. H. Ying Huang, “Outfit recommendation based on deeplearning,” Advances in Computer Science Research, \\ volume 74 2nd International Conference on Computer Engineering, Information Science & Application Technology (ICCIA 2017), Atlantis-Press. [4] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778. [5] Gregory F. Cooper, and Edward Hershkovits, “A Bayesian Method for the Induction of Probabilistic Networks from Data,” MACHINE LEARNING, VOL. 9,pp.309-347,1992 . [6] Claypool, M., Gokhale, A., Miranda, T., Murnikov, P., Netes, D. & Sartin, M. (1999). Combing content-based and collaborative filters in an online newspaper. Proc. of Workshop on Recommender Systems-Implementation and Evaluation, 1999, pp. 6-22. [7] Guo, G., Zhang, J. & Thalmann, D. (2014). Merging trust in collaborative filtering to alleviate data sparsity and cold start. Knowledge-Based Systems, 57, pp. 57-68. [8] Iglesias, F. & Kastner, W. (2013). Analysis of similarity measures in times series clustering for the discovery of building energy patterns. Energies, 6(2), pp. 579-597. [9] Isinkaye, F. O., Folajimi, Y. & Ojokoh, B. A. (2015). Recommendation systems: Principles, methods and evaluation. Egyptian informatics journal, 16, pp. 261-273. [10] Jannach, D. & Friedrich, G. (2013). Tutorial: recommender systems. International Joint Conference on Artificial Intelligence Beijing, 2013, pp. 1-144. [11] Y Qian, P Giaccone, M Sasdelli and E Vazquez,”Algorithmic clothing: hybrid recommendation, from street-style-to-shop” , Nov 2017 [12] Mohamed Elleuch, Anis Mezghani, Mariem Khemakhem, Monji Kherallah “Clothing Classification using Deep CNN Architecture based on Transfer Learning” ,2021 DOI:10.1007/978-3-030-49336-3_24 [13] Saurabh Gupta, Siddartha Agarwal, Apoorve Dave. “Apparel Classifier and Recommender using Deep Learning.” (2015). [14] Bossard, Lukas, Matthias Dantone, Christian Leistner, Christian Wengert, Till Quack and Luc Van Gool. “Apparel Classification with Style.” ACCV (2012). [15] Krizhevsky, Alex, Ilya Sutskever and Geoffrey E. Hinton. “ImageNet classification with deep convolutional neural networks.” Communications of the ACM 60 (2012): 84 - 90. [16] Congying Guan, Shengfeng Qin, Yang Long, (2019) \\\"Apparel-based deep learning system design for apparel style recommendation\\\", International Journal of Clothing Science and Technology.

Copyright

Copyright © 2024 Swapnil Thakur, Shreyas Dixit, Prof. Rahul Dagde. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.